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4.  Purpose: 

The  purpose  of  this  research  project  is  to  determine  the  feasibility  and  utility  of  Whole  Slide  Imaging  (WSI)  and 
Robotic  Microscopy  in  a  number  of  pathology  laboratory  environments.  Our  research  project  involves  three 
phases.  Phase  I  evaluates  the  utility  and  feasibility  of  a  WSl-Based  Educational  Slide'  lmage  Collection.  Phase  2 
evaluates  inter-  and  intra-facility  Quality  Assurance  (QA)  practices  using  WSI.  Finally,  Phase  3  compares  the 
utility  and  feasibility  of  real-time  robotic  microscopy  for  consultations  to  that  of  WSI. 


5.  Status  of  the  Study.  Mark  the  status  of  the  study  (a-e). 

a.  _ Active  with  ongoing  data  collection.  Request  approval  to  remain  open. 

b.  _ Active  with  data  collection  complete.  Request  approval  to  remain  open. 

e.  _  Study  was  never  initiated  and  request  termination  of  the  study. 

d.  X  Completed,  research  implemented  and  results  available.  Request  approval  to  close. 

e.  _  Inactive,  protocol  never  initiated,  but  want  to  keep  in  open.  Request  approval  to  remain  open. 

6.  Summary  of  Progress:  This  report  covers  the  following  period  of  time:  April  2007  to  January  2009. 
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statistically  significant 
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14.  ABSTRACT 

The  purpose  of  this  research  project  is  to  determine  the  feasibility  and  utility  of  Whole  Slide  Imaging 
(WSI)  and  Robotic  Microscopy  in  a  number  of  pathology  laboratory  environments.  Our  research  project 
involves  three  phases.  Phase  1  evaluates  the  utility  and  feasibility  of  a  WSI-Based  Educational  Slide/Image 
Collection.  Phase  2  evaluates  inter-  and  intra-  facility  Quality  Assurance  (QA)  practice  using  WSI.  Finally, 
Phase  3  compares  the  utility  and  feasibility  of  real-time  robotic  microscopy  for  consultation  to  that  of  WSI. 
The  Telepathology  projects  have  been  successfully  implemented  and  results  are  encouraging.  Primary 
goals  include  evalaluation  of  WSI  and  robotic  microscopy  as  potential  platforms  for  USAF  pathology. 
USAF  advocates  and  leaders  for  digital  pathology  are  working  to  strategize  integration  of  digital  pathology 
into  the  AFMS.  It  is  hypothesized  that  digital  slides  can  be  effectively  implemented  within  existing 
workflows  and  will  be  useful  in  establishing  timely  inter-facility  diagnoses  and  consultation  across 
multi- facility  health  systems.  In  summary,  current  Telepathology  systems,  especially  whole  slide  imaging, 
are  capable  of  providing  useful  levels  of  surgical  pathology  reviews  across  distributed  health  systems  and 
they  will  only  improve  over  time. 
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»»h;,sc  I:  Whole  Slide  Image  (WSl)  Based  Educational  Slide  Collection 

The  University  of  Pittsburgh  Medical  Center’s  (UPMC)  Innovative  Medical  Information  and  Technologies  (IMITS) 
Center  has  developed  an  educational  database  of  over  200  digital  images  of  interesting  pathology  cases  As  part  of 
the  Evaluation  Project,  a  Needs  Assessment  survey  (pre-implementation)  was  distributed  across  approved  Air  Force 
Bases  to  determine  the  need  and  feasibility  of  an  educational  database  named  PathEd  This  questionnaire  evaluated 
current  capabilities  of  Telepathology  as  well  as  cunent  practices.  The  Evaluation  Team  received  twelve  responses 
from  Air  Force  Pathologists.  Below  is  a  table  containing  a  summary  of  base  response  rates. 


Air  Force  Base 

Number  of  Rcvjionvcs_ 

Lackland  AFB 

" 

Offutt  AFB 

1 

Travis  AFB 

1 

Eglin  AFB 

1 

In  order  to  evaluate  user  satisfaction  and  other  parameters,  the  evaluation  team  developed  a  12  question  assessment 
questionnaire  (Post-implementation  Qucstionnairc/lnterview)  which  was  to  be  incorporated  into  the  educational 
database.  It  was  to  appear  after  the  subject  has  viewed  five  cases.  This  questionnaire  was  to  be  voluntary,  and  if 
after  the  participant  had  declined  three  times  to  complete  the  questionnaire,  the  questionnaire  would  not  present 
itself  again.  It  was  not  wntten  into  the  programming  for  PathEd  and  thus  was  never  launched 

However,  in  the  spring  of  2008,  UPMC  resident  pathologists  were  asked  to  complete  a  survey  (PathEd  Application 
Surv  ey)  that  assessed  the  ease  of  use  of  the  PathEd  application.  This  information  was  used  to  provide  feedback  to 
the  UPMC  PathEd  development  team.  As  a  result  of  this  survey,  many  issues  were  identified  and  resolved  and 
suggestions  were  incorporated  into  the  applications. 


Phase  2:  Inter-  and  Intra-facilitv  WSl  Quality  Assurance  Study 

Case  Selection 

A  CoPuth  (Cemer  Corporation,  Kansas  City.  MO)  laboratory  information  system  (I. IS)  electronically  queried  its 
surgical  pathology  database  for  all  cases  in  2004  that  had  been  previously  viewed  for  quality  assurance.  This  initial 
query  resulted  in  1 150  possible  cases.  For  research  purposes,  investigators  decided  to  exclude  cases  that  had  over 
five  blocks.  This  exclusion  factor  resulted  in  497  cases  available  for  study  purposes.  The  computer  selected  every 
tenth  case  from  this  pool  until  thirty  cases  were  selected  I  he  thirty  cases  selected  resulted  in  67  diagnosable  parts, 
comprised  of  202  slides.  De-identified  pathology  reports  and  slides  of  the  thirty  cases  were  obtained  and  used  for 
the  study. 

Digital  Slide  Preparation  and  View  ing 

Glass  slides  of  the  thirty  surgical  pathology  cases  w  ere  scanned  using  an  Aperio  T2  scanner.  Che  T2  scanner  used  a 
20x  microscope  objective  and  had  a  spatial  sampling  period  of  0.47  pm  per  pixel.  The  scanner  was  equipped  with 
dual  2.4  GHz  Xeon  processors  with  4  GB  of  RAM.  Digital  slides  were  placed  on  a  server  with  dual  3  GHz  Xcon 
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processors  and  4  GB  of  RAM  running  Microsoft  Windows  Server  2000.  The  digital  slides  were  viewed  using 
Aperio’s  Spectrum  webviewer.  Aperio’s  webviewer  could  be  accessed  from  any  computer  that  the  pathologist  had 
access  to.  Typical  computers  had  Pentium  processors  w  ith  5 12  to  2048  megabytes  RAM.  Microsoft  Windows 
2000/XP,  and  connectivity  to  the  internet. 

Participant  Recruitment 

Interested  pathologists  were  identified  on  USAF  bases  and  within  UPMC.  All  sites  were  IRB/SGR  approved.  If 
possible,  project  personnel  met  with  the  pathologists  directly  to  personally  discuss  the  project  in  detail  and  to 
confirm  participation.  A  recruitment  handout  and  tri-fold  were  developed  and  sent  with  a  letter  or  email  to  eligible 
pathologists  at  both  UPMC  and  USAF  bases.  Six  pathologists  were  recruited  from  multiple  UPMC  sites. 

Data  Collection 

We  collected  data  at  three  time  points  during  the  study.  The  first  time  point  occurred  prior  to  the  study.  A 
questionnaire  was  designed  that  asked  pathologists  about  their  pathology  experience,  foreseeable  roles  of  digital 
pathology,  potential  obstacles,  and  digital  slide  familiarity. 

The  second  data  collection  time-point  was  during  the  analysis  of  digital  slides.  Participants  were  asked  to  complete 
slide  and  case  surveys.  For  slide  surveys,  variables  such  as  focus  quality,  artifact  presence,  and  tissue  component 
(nuclear,  cytoplasmic,  red  blood  cell,  non-cellular,  vascular,  and  lymphocytic  detail)  focus  quality  were  recorded. 
Focus  quality  and  tissue  component  focus  quality  used  a  five  point  Likert  scale  that  ranged  from  excellent  to  very 
poor.  “Excellent"  focus  quality  ratings  indicated  a  flaw  less  image  both  in  color  and  in  sharpness.  A  rating  of 
“Good"  meant  nearly  flawless  with  adequate  sharpness.  “Fair"  quality  meant  that  25%  to  50%  of  the  image  was  out 
of  focus,  but  evaluation  of  the  digital  slide  is  still  possible.  “Poor"  ratings  indicated  that  most  of  the  slide  was  out  of 
focus,  and  it  prevented  diagnosis.  “Very  Poor"  meant  that  the  entire  slide  was  out  of  focus,  which  precluded  a 
diagnosis.  Image  response  speed,  apparatus  ease  of  use,  and  internet  stability  were  also  recorded.  These  variables 
used  a  low,  medium,  and  high  scale  as  perceived  by  the  pathologist.  The  slide  surveys  incorporated  in  an  electronic 
time  stamp  that  determined  the  relative  time  to  complete  the  evaluation  of  a  slide. 

Case  surveys  were  developed  in  order  to  record  the  level  of  diagnostic  agreement.  After  analyzing  every  slide  for  a 
biopsied  part,  participants  would  view  the  original  diagnosis  and  rate  their  agreement  level  using  a  five  point  Likert 
scale.  A  rating  of  “Complete  Agreement"  meant  that  all  aspects  of  the  original  diagnosis  were  agreed  upon.  A 
“Reporting  Error"  rating  meant  there  was  a  typographical  error  in  the  original  report,  but  it  did  not  affect  patient 
management  or  prognosis.  “Mild  Disagreement"  indicated  a  disagreement  in  the  subtype  of  diagnosis.  For 
example,  the  wrong  type  of  sarcoma  was  reported.  “Moderate  Disagreement”  rankings  indicated  a  disagreement,  but 
it  did  not  affect  patient  management  or  prognosis.  Finally,  a  rating  of  “Marked  Disagreement”  indicated  a 
disagreement  with  the  original  diagnosis  that  affected  patient  prognosis  and/or  management.  Diagnostic  agreement 
was  rated  for  every  diagnostic  part  of  every'  case.  The  smallest  cases  had  one  biopsied  part,  w  hile  the  largest  case 
had  ten  biopsied  parts.  Cases  that  received  any  disagreement  rating  were  reviewed  by  the  principal  investigator  in 
order  to  determine  if  the  original  pathologist  needed  notification.  Case  surveys  also  recorded  perceived  case 
complexity',  diagnostic  confidence,  and  time-to-complclc.  Case  surveys  had  a  timestamp  incorporated  into  them  that 
tracked  case-completion  values. 

Once  participants  completed  case  and  slide  surveys,  they  were  asked  to  complete  a  post-study  questionnaire.  This 
questionnaire  recorded  responses  to  variables  such  as  confidence  in  use  of  digital  pathology  technology,  strengths 
and  weaknesses  of  digital  slides,  digital  slide  feasibility,  and  the  role  of  digital  slides  in  the  every  day  practice  of 
pathology. 

Data  was  collected  using  a  Microsoft  Access  database.  Forms  were  designed  to  help  the  participant  matriculate 
through  the  study  and  provided  an  interface  where  pathologists  could  enter  their  own  responses.  Participants  could 
submit  their  data  with  a  click  of  a  mouse  to  the  study's  honest  broker. 

We  have  prepared  one  paper  which  discusses  the  use  of  the  Microsoft  Access  database  as  an  evaluation  tool.  This  is 
only  now  in  DRAFT  form  and  will  be  submitted  to  an  evaluation  or  pathology  journal. 
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Phase  3:  Robotic  Microscopy  vs.  Whole  Slide  Imaging  For  Real  Time  Diagnosis 

Case  Selection 

Forty  cases  were  selected  from  UPMC  hospitals.  The  diagnostic  intraoperative  consultation  (frozen  section)  slides 
were  selected  by  the  principal  investigator.  Intraoperative  consultation  is  an  important  component  in  the  practice  of 
surgical  pathology.  Digital  pathology  technology  may  be  useftil  in  situations  where  a  local  pathologist  is  not 
available.  Tissues  analyzed  during  intraoperative  consultation  are  often  thicker  than  the  permanent  tissue 
preparations,  therefore,  these  tissue  samples  may  present  challenges  to  current  digital  slide  technologies  in  providing 
pathologists  w  ith  diagnostic  grade  images 

Digital  Slide  Preparation  and  Viewing 

Whole  Slide  Images  were  created  using  a  Trestle, 'Zeiss  fifty  slide  loading  apparatus  paired  w  ith  a  Olympus  BX5 1 
microscope  equipped  w  ith  a  Jai  3CCD  RGB  camera.  These  digital  slides  were  placed  on  a  machine  equipped  with 
dual  3  GHz  Xeon  processor  with  4  GB  of  RAM  and  Microsoft  Windows  2000  Server. 

Pathologists  used  an  Olympus  BX-41  microscope  equipped  with  a  Jai  M7  camera.  Digital  slides  viewed  w  ith 
robotic  microscopy  and  whole  slide  imaging  used  Trestle's  Medmicro  software.  Data  was  collected  using  a 
Microsoft  Access  database. 

Participant  Recruitment 

Interested  pathologists  were  identified  on  USAF  bases  and  within  UPMC.  All  sites  were  IRB/SGR  approved.  If 
possible  project  personnel  met  with  them  personally  to  discuss  the  project  in  detail  and  to  confirm  participation.  A 
recruitment  handout  and  tri-fold  were  developed  and  sent  with  a  letter  or  email  to  eligible  pathologists.  In  February 
of  2008  a  Digital  Pathology  Training  Seminar  was  held  at  UPMC.  This  allowed  time  for  UPMC  and  USAF 
pathologists  to  meet  and  work  together  on  the  evaluation  study.  At  that  time  we  were  able  to  solicit  participation 
from  three  USAF  bases,  one  VA.  and  one  UPMC  pathologist.  They  were  able  to  complete  or  complete  in  part  the 
robotic  microscopy  portion  of  the  study.  Our  total  sample  included  six  UPMC  and  four  military  pathologists. 

Data  Collection 

We  collected  data  at  three  time  points  during  the  study.  The  first  time  point  occurred  prior  to  the  study.  A 
questionnaire  was  designed  that  asked  pathologists  if  they  had  ever  used  either  WS1  or  robotic  microscopy.  It  also 
assessed  their  pathology  experience,  opinions  on  foreseeable  roles  of  digital  pathology  and  potential  obstacles. 

The  second  data  collection  was  accomplished  during  the  analysis  of  slides.  Participants  were  asked  to  complete  case 
properties  surveys.  They  w  ere  asked  to  rate  the  image  on  a  four  point  scale;  “Excellent"  focus  quality  ratings 
indicated  a  flawless  image  both  in  color  and  in  sharpness.  A  rating  of  “Good"  meant  nearly  flawless  with  adequate 
sharpness.  “Fair"’  quality  meant  that  25%  to  50%  of  the  image  was  out  of  focus,  (i.e.,  distortion  makes  evaluation 
difficult  or  lack  of  quality  impedes  the  ability  to  render  a  diagnosis).  “Poor”  ratings  indicated  that  most  of  the  slide 
was  out  of  focus,  and  it  prevented  diagnosis.  The  pathologists  w  ere  also  asked  to  provide  a  diagnosis.  They  were 
also  asked  their  opinion  on  the  use  of  use  of  the  apparatus,  complexity  of  the  case  and  confidence  in  their  diagnosis. 

Once  participants  completed  case  properties  surveys,  they  were  asked  to  complete  a  post-study  questionnaire.  This 
questionnaire  recorded  responses  to  variables  such  as  major  strengths  of  either  WSI  or  robotic  microscopy, 
confidence  of  diagnosis  and  if  they  had  a  preference  between  the  two  technologies. 

Data  was  added  to  a  Microsoft  Access  database.  Forms  were  designed  to  help  the  participant  matriculate  through 
the  study  and  provided  an  interface  where  pathologists  could  enter  their  own  responses.  Participant  could  submit 
their  data  with  a  click  of  a  mouse  to  the  study’s  honest  broker. 

We  plan  to  subsequently  publish  results  in  an  appropriate  journal. 

b.  For  the  entire  study: 

Digital  pathology  QA  controlled  trial  studies  were  conducted  at  UPMC  in  FY04,  and  the  results  of  those  studies 
affirm  the  reliability  and  usefulness  of  WSI  in  the  clinical  setting.  The  goal  of  the  evaluation  projects  was  to 
determine  the  utility  of  inter-facility  QA  based  on  WSI  technology  within  the  AFMS. 
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Whole  slide  imaging  is  a  method  of  scanning  slides  and  posting  the  digitized  images  on  servers  and  transmining 
them  over  computer  networks  This  process  permits  independent  viewing  of  images  by  large  numbers  of  people  in 
diverse  locations.  It  involves  a  fusion  of  microscopy  and  digital  technologies.  Since  these  image  files  tend  to  be 
quite  large,  they  are  usually  served  up  as  •‘tiles”  transmitted  just  in  time  as  the  viewer  needs  them;  only  the  desired 
tiles  are  actually  transmitted.  Advances  in  digital  technologies  make  it  possible  to  view  images  of  slides  with 
resolutions  nearing  those  visible  under  an  optical  microscope.  Image  quality  remains  an  issue  for  consumers, 
however,  and  scanning  manufacturers  continuously  strive  to  make  improvements.  These  improvements  increasingly 
focus  on  changes  that  will  impact  the  integration  of  these  systems  into  the  clinical  pathology  environment.  Scanners 
are  expensive  and  vary  in  performance.  Rigorous  assessments  of  system  features  and  their  fitness  for  today  's 
medical  environments  are  required. 

c  If  this  is  a  FINAL  REPORT: 

1.  Were  the  protocol  objectives  met  and  how  will  the  outcome  benefit  the  DoD/USAF? 

•  Phase  1:  Create  a  WSI-Based  Educational  Slide/lmngc  Collection 

Typical  pathology  education  includes  providing  slide  sets  in  a  given  domain  for  students  to  diagnose.  There  are 
many  obvious  limitations  with  this  form  of  education.  A  few  include:  cases  with  little  information;  difficult  access  to 
slides,  especially  when  slides  must  be  shared  among  students;  and  instructor  expectations  that  students  can 
determine  the  pertinent  diagnostic/prognostic  features  that  lead  to  a  diagnosis.  PatliEd  was  designed  to  be  used  to 
augment  pathology  education  for  USAF  and  UPMC  pathologists  and  pathology  students,  addresses  the  limitations 
of  typical  pathology  education  PatliEd  allows  input  of  any  relevant  case  information  including  information  from 
the  pathology  report  and  added  metadata  such  as  Category  and  Subcategory.  With  each  case  posted,  additional 
electronic  files,  such  as  PDFs  and  JPEGs  can  also  be  uploaded  to  the  case.  PathEd  uses  digital  slides  scanned  from  a 
variety  of  whole  slide  imaging  systems  w  ith  digital  slides  organized  in  a  tree  based  on  the  case's  Organ.  Category 
and  Subcategory.  It  provides  instructors  with  the  ability  to  annotate  digital  slides  and  highlight  pertinent  case 
features  and  offers  instructors  the  option  to  create  quizzes,  w  hich  may  be  linked  to  digital  slides  or  cases,  for 
assessment  of  the  students'  comprehension. 

To  date,  over  200  de-ideniified  cases  have  been  uploaded  to  the  repository  and  are  ready  for  viewing  by  the 
pathologists.  Feedback  from  initial  users  has  improved  layout  of  the  interface  and  the  performance  of  system.  It  is 
anticipated  that  users  w  ill  continue  to  provide  feedback  that  will  be  used  to  shape  PathEd's  features  and  capabilities. 
PathEd  applications  can  readily  be  expanded  to  encompass  other  domains  and  support  training  for  other  allied  health 
professionals. 

•  Phase  2:  Conduct  WSI-Based  Inter-Facility  Quality  Assurance  (QA) 

This  project  extended  the  FY04  IMITS  Tclepathology  QA  studies  conducted  at  UPMC  to  include  members  of  tire 
USAF  network  of  pathologists  The  goal  of  this  project  was  to  determine  the  utility  of  inter-facility  QA-based  WSI 
technology  Inter-facility  WSI  QA  may  allow  for  more  cost  effective,  higher  quality  QA  procedures  in  the  USAF 
system. 

WSI  technology  is  still  relatively  new;  the  first  digital  slide  scanner  were  built  in  2000  and  the  technology  is  rapidly 
improving  For  automated,  high-speed  WSI  to  reach  its  potential  in  pathology  practice,  it  must  undergo  continuous 
evaluation  and  validation  by  the  pathology  community.  During  the  FY04  Tclepathology  Project,  UPMC  established 
a  protocol  for  assessing  image  quality  and  clinical  functionality  of  advanced  image  scanning  systems.  At  the 
conclusion  of  the  FY04  Tclepathology  Project,  results  from  a  scries  of  controlled  clinical  validations  studies 
indicated  that  whole  slide  image  information  is  sufficient  for  pathologists  to  make  reliable  diagnostic  decisions. 

These  published  studies  have  contributed  to  the  acceptance  of  digital  pathology  in  clinical  prncticc;  however,  more 
research  is  needed  to  better  assess  clinical  capabilities. 

The  following  controlled  clinical  validation  studies  were  completed  during  the  FY05  project 

El  ALVA  nos  OF  H  HOLE  SLIDE  IMAGE  IMMVSOHISTOCIIEMISTR I  (IHC )I\TERPRETA  I  ION  IN  CHALLENGING 

r rost a  te  Needle  biopsies 
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The  IHC  study  was  designed  to  lest  the  pathologist’s  ability  to  interpret  IHC  staining  in  prostate  needle  biopsies  by 
comparing  automated  WSI  w  ith  traditional  glass  slide  microscopy  Additionally  ,  and  perhaps  more  importantly  ,  the 
study  tested  the  potential  application  of  WSI  across  geographically  dispersed  health  systems,  such  as  UPMC.  which 
has  a  single  central  IHC  lab  that  serves  many  hospitals.  The  study  took  place  in  the  Division  of  Anatomic  Pathology 
at  UPMC  (primarily  at  Shadysidc  and  Presbyterian  Hospitals)  and  included  evaluators  from  UPMC  and  Johns 
Hopkins  University.  This  retrospective  study  evaluated  the  interpretation  of  II  1C  stains  performed  in  difficult 
prostate  biopsies  using  WSI.  The  study  included  30  foci  with  IHC  stains  identified  by  the  original  pathologist  as 
both  difficult  and  pivotal  to  the  final  diagnosis.  WSI  were  created  from  the  glass  slides.  An  evaluation  form  was 
designed  to  capture  data  in  two  phases:  1 )  interpretation  of  whole  slide  images,  and  2)  interpretation  of  glass  slides 
Data  included  stain  interpretations,  diagnoses,  and  other  parameters  such  as  time  required  to  diagnose  and 
image  slide  quality  WSI  diagnostic  validity  was  almost  perfect’  for  one  pathologist,  ‘substantial'  for  three 
pathologists,  and  ‘moderate'  for  one  pathologist  Except  for  one  instance,  WSI  technology  was  not  felt  to  be  the 
cause  of  disagreements.  Data  w  as  also  collected  from  an  expert  prostate  pathologist.  Diagnostic  agreement  betw  een 
the  final  consensus  diagnoses  of  the  group  and  those  of  the  domain  expert  was  ‘almost  perfect’.  These  results  are 
encouraging  and  compare  favorably  with  other  efforts  to  quantify1  diagnostic  variability  in  surgical  pathology1.  We 
concluded  that  WSl-based  technology  can  currently  permit  accurate  interpretation  of  IHC  stains  in  the  setting  of 
diagnostically  difficult  prostate  biopsies.  Comparison  of  a  consensus  diagnosis  to  a  “gold-standard"  expert  opinion 
also  confirmed  well-established  fact  that  certain  difficult  cases  may  defy  confident  diagnosis  by  a  single  pathologist, 
even  with  glass  slides.  These  findings  provide  evidence  supporting  the  validity  of  WSI  technology  for  interpretation 
of  IHC  in  prostate  biopsies,  and,  further,  these  findings  suggest  that  development  of  other  “virtual"  IHC  applications 
is  feasible  and  should  be  pursued  (including  relevant  specific  validation  studies).  Results  subsequently  published  in 
Human  Pathology. 

Frozen  Section  examination  Using  WSI  for  quality  assurance 

The  use  of  frozen  sections  or  intra-operative  consultstions  is  an  important  part  of  surgery,  providing  critical 
guidance  to  the  surgeon  about  margin  status  and  or  tumor  type  w  hile  the  patient  is  still  on  the  operating  table. 

UPMC  researchers  assessed  the  use  of  W’SI  for  evaluation  of  frozen  section  specimens.  Five  pathologists  compared 
the  concordance  between  intra-operative  and  final  diagnosis  using  WSI  technology.  Twenty  four  entire  cases 
(including  47  surgical  parts  and  391  slides)  from  the  genitourinary  surgical  bench  were  selected  randomly  and  were 
scanned  by  Aperio  T2  WSI  system  Pathologists  diagnosed  cases  using  glass  slides  or  WSI.  During  examination  of 
the  specimens,  pathologists  completed  the  standard  QA  form  and  evaluated  diagnostic  concordance,  confidence, 
case  complexity,  and  time  to  complete  the  case.  After  six  w  eeks,  approximately  20%  of  the  cases  w  ere  assigned  to 
the  same  pathologists  for  intra-observer  reliability  assessment 

Results  indicated  that  the  diagnostic  confidence  for  both  the  glass  and  WSI  interpretation  was  high.  Fifteen  cases 
with  234  images  were  evaluated  across  both  conditions  w  ith  12  reported  as  high  confidence  and  3  reported  as 
medium  confidence.  There  were  no  disagreements  by  the  pathologists.  The  diagnosis  remained  unchanged  after 
reviewing  the  corresponding  WSI  or  glass  slides  in  all  15  of  these  cases  which  were  also  concordant  with  the 
primary  diagnosis  in  original  pathology  report. 

This  study  helped  refine  research  methodologies  that  were  applied  to  other  studies.  The  study  indicated  that 
diagnostic  confidence  as  well  as  intra-observer  reliability  for  the  WSI  appeared  high  in  the  cases  selected  for  this 
assessment.  And.  although  the  study  was  done  on  a  small  number  of  cases,  the  results  were  encouraging  and 
support  WSI  as  a  useful  tool  in  pathology  practices,  especially  for  QA  studies  in  surgical  pathology  Study  results 
were  subsequently  presented  in  both  an  oral  and  poster  presentation. 

Frozen  Section  Quality  Assurance  Pilot  Stvdy 

Twenty  difficult  cases  were  selected  by  a  pathologist  w  ho  did  not  participate  in  the  reading  of  the  slide  images.  All 
slides  associated  with  the  cases  were  scanned  using  the  Aperio  Scanscope  T2.  A  total  of  1 84  slides  were  scanned. 
Four  surgical  pathologists  volunteered  to  participate  in  the  QA  study.  A  survey  was  developed  that  recorded  the 
pathologists’  perceptions  of  diagnosis,  image/slidc  quality,  case  complexity  ,  diagnostic  confidence,  and  time  to 
complete  case.  The  study  design  required  each  participant  to  view  10  cases  via  WSI  and  10  coses  via  light 
microscopy.  The  10  cases  viewed  via  light  microscopy  were  different  than  the  cases  viewed  via  WSI.  Each 
participant  scheduled  an  hour  of  time  to  complete  the  study,  and  no  two  pathologists  participated  in  the  study  at  the 
same  time.  A  member  of  the  evaluation  team  used  a  stop  watch  to  measure  the  amount  of  time  spent  on  each  case 
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The  WSI  portion  of  the  project  was  not  completed  by  any  of  the  participating  pathologists  because  of  a  number  of 
problems  relating  back  to  study  design  and  implementation.  Study  design  issues  included  case  selection,  lack  of 
“telepathology”  training,  incorrect  labeling  of  slides  images,  and  ambiguity  of  frozen  section  parts.  The  project  was 
halted  by  the  University  of  Pittsburgh  Principal  Investigator.  However,  many  lessons  were  learned  during  this  pilot 
study  with  design  improvements  incorporated  into  future  evaluation  projects. 

Conduct  WSI-Based  Inter-Facility  Quality  Assurance  (QA) 

Study  design  and  data  collection  have  already  been  discussed  in  detail  in  section  6.  Summary  of  Progress.. 
Preliminary'  results  were  presented  at  the  American  Telemedicine  Association  13"1  Annual  International  Trade 
Meeting  and  Exposition.  April  6-8,  2008.  Seattle,  WA.  We  have  prepared  one  paper  which  discusses  the  use  of  the 
Microsoft  Access  database  as  an  evaluation  tool.  This  is  only  now  in  DRAFT  form  and  will  be  submitted  to  an 
evaluation  or  pathology  journal. 

•  Phase  3:  Compare  Robotic  Microscopy  w  ith  WSI  for  Real  Time  Consultations 
Telepathology  allows  microscopic  images  of  stained  tissue  specimens  and  other  forms  of  visual  data  to  be  shared 
among  pathologists  for  diagnosis,  quality'  assurance  and  training,  regardless  of  physical  location.  Through  the 
1M1TS  Telepathology  Project,  UPMC  is  helping  the  AFMS  create  an  enterprise  Telepathology  network.  Network 
security  certifications  are  now  in  place  for  the  robotic  microscope  and  will  be  secured  for  whole  slide  imaging 
systems  in  the  near  future. 

This  is  happening  at  a  time  when  the  need  for  prompt  diagnostic  services  in  the  military  is  increasing.  Combat 
environments  are  experiencing  threats  for  the  release  of  biological  and  infectious  agents  that  could  potentially 
cripple  our  forces.  Homeland  security  has  identified  similar  threats  here  in  die  U.S.  In  addition,  die  military  must 
be  prepared  to  respond  to  natural  catastrophes,  such  As  Hurricane  Katrina,  widi  diagnostic  services  that  can  aid  in 
recovery'  and  disease  management. 

Through  project  funding,  four  robotic  microscopes  were  ordered  for  USAF  locations  in  late  December  2006.  It  was 
anticipated  diat  diese  systems  would  be  deployed  at  identified  USAF  locations  early  in  2007.  The  rigors  of  the 
DITSCAP  approval  processes,  however,  extensively  delayed  system  installations.  D1TSCAP  certification  was 
finally  achieved  in  September  2007.  Installation  then  continued  to  be  delayed  pending  USAF  Communications 
Agency  signed  Authority  to  Connect.  AtC  was  acquired  in  March  2008.  Base-level  approval  and  configuration  was 
required  next  before  installations  could  be  scheduled.  Robotic  microscopes  installations  at  all  four  USAF  locations 
were  completed  in  September  2008. 

Although  the  static  image  and  robotic  systems  are  basic  to  a  Telepathology  network,  both  systems  have  limited 
applications  and  built-in  constraints.  The  extension  of  these  technologies  into  every  day  practice  is  restricted.  Static 
image  systems  have  numerous  restrictions  but  perhaps  the  most  important  is  that  single  field,  camera-on-microscope 
systems  cannot  document  the  entire  slide  and  forces  the  pathologist  to  spend  valuable  time  finding  and  capturing 
limited  fields  of  interest  and  therefore  necessitates  highly  trained  individuals  on  both  ends. 

Robotic  systems  have  proven  to  be  successful  in  promoting  digital  pathology  and  they  are  expected  to  be  of  value  in 
the  USAF*  environment.  They  are,  however,  real-time,  remote  control  systems  that  require  hands-on  involvement  at 
the  scope  site  with  real-time  coordination  by  the  remote  pathologist  on  the  diagnosing  side.  As  part  of  the  core 
operations,  robotic  systems  do  not  provide  permanent  image  storage. 

Whole  slide  imaging  systems  are  the  most  promising  for  advancing  digital  pathology  within  the  AFMS  and 
elsewhere.  These  systems  represent  a  series  of  technologies  for  capturing  an  entire  glass  microscopic  slide  at  high 
resolution,  storing  this  image  on  a  server,  and  displaying  the  image  on  demand  over  a  broadband  network.  Like 
robotic  systems,  The  user  has  access  to  the  entire  histologic  section  and  can  “pan  and  zoom"  across  the  “digital  slide" 
as  desired,  thus  simulating  the  exam  carried  out  on  the  microscope.  Like  static  image  systems,  the  whole  slide 
image  is  persistent,  can  be  archived  for  future  use,  and  can  be  integrated  with  other  electronic  data  (such  as 
pathology  reports  or  laboratory  information  systems).  Most  importantly,  because  whole  slide  image  capture  is  done 
automatically  by  robotic  scanners,  it  can  be  done  “behind  the  scenes”  and  does  not  require  technical  imaging  skills 
on  the  part  of  the  pathologist.  It  is  more  important  than  ever  that  we  continue  to  support  this  initiative. 
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For  the  USAF  to  realize  the  potential  of  digital  pathology  technologies,  we  recommend  the  purchase  of  whole  slide 
imaging  scanning  systems  for  each  major  medical  facility  within  the  AFMS.  With  a  suitable  network  of  digital 
systems  in  place,  research  and  development  activities  can  be  conducted  to  advance  know  ledge  of  digital  pathology 
applications  and  practices,  particularly  within  the  military. 

In  the  emerging  w  orld  of  digital  pathology,  it  is  important  to  have  a  definitive  study  that  compares  side-by-side  real¬ 
time  robotic  Telepathology  with  WSl-based  Telepathology.  The  goal  of  this  evaluation  project  was  to  determine  the 
utility  of  robotic  microscopy  for  real-time  consultation  in  the  USAF,  and  compare  it  with  the  utility  of  WSI 
consultation.  The  University  of  Pittsburgh  Evaluation  Team  designed  a  research  study  to  compare  the  merits  of 
robotic  microscopy  systems  with  whole  slide  imaging  systems. 

Study  design  and  data  collection  have  already  been  discussed  in  detail. 

We  plan  to  subsequently  publish  results  in  an  appropriate  journal. 

2.  Protocol  Outcomes  Summary: 

The  Telepathology  projects  have  been  successfully  implemented  and  results  are  encouraging.  Primary'  goals  include 
evaluation  of  WSI  and  robotic  microscopy  as  potential  platforms  for  USAF  pathology.  USAF  advocates  and  leaders 
for  digital  pathology  are  working  to  strategize  integration  of  digital  pathology  into  the  AFMS.  It  is  hypothesized 
that  digital  slides  can  be  effectively  implemented  within  existing  workflows,  and  will  be  useful  in  establishing 
timely  inter-facility  diagnoses  and  consultations  across  multi-facility  health  systems.  In  summary,  current 
Telepathology  systems,  especially  whole  slide  imaging,  are  capable  of  providing  useful  levels  of  surgical  pathology 
reviews  across  distributed  health  systems  and  they  will  only  improve  over  time. 


◄  IF  THIS  IS  A  FINAL  REPORT  PROCEED  TO  #  9  ► 

7.  Protocol  Changes: 

a.  _  No  changes  are  anticipated  and  the  project  will  continue  as  prev  iously  approved  by  the  IRB. 

b.  _  Changes  arc  anticipated  as  described  below:  (Description . t 

c.  When  do  you  anticipate  PCSing  or  separating? _  (Insert  Date) 

8.  Protocol  Personnel  Changes: 

Has  there  been  any  Principal  or  Associate  Investigator  (PI/A!)  changes  since  approval  of  protocol  or  the  last 

continuation  review? _ Yes _ No.  If  yes,  complete  the  following  sections  (Additions/Deletions).  For  Pl/AI 

changes,  indicate  whether  or  not  the  IRB  approved  this  change. 

a.  Additions: 

b.  Deletions: 


9.  Status  of  Approved  Funding: 

a.  Funding  from  the  Surgeon  General  Office  (SGO)  in  the  amount  of  S  0  was  approved  in  my 

original  protocol.  I  have  utilized  $ _ 0 _ of  these  funds  to  date. 

b.  Request  funding  in  the  amount  of  $  0  for  FY  09  in  addition  to  amount  originally  approved. 

Describe  or  itemize  here,  or  attach  a  separate  sheet. 

c.  I  have  received  External  Resources  to  support  this  study  in  the  form  of:  (describe  all  those  applicable: 
loaned  equipment,  consumable  supplies,  drugs  from  a  non-DoD  source,  and/or  funds  from  an  external  source,  in  this 
case  give  the  name  and  amount) 
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The  UPMC  work  is  supported  by  funding  from  the  U.S.  Air  Force  administered  by  the  U.S.  Army  Medical 
Research  Acquisition  Activity  (USAMRAA).  820  Chandler  Street,  Fort  Detrick  MD  21702-5014,  Contract 
No.  DAM D 17-03-2-00 17.  Funding  provided  by  AF'SGR  is  from  a  withhold  taken  from  the  appropriation 
for  this  purpose  prior  to  awarding  this  contract.  The  content  of  the  information  does  not  necessarily  reflect 
the  position  or  policy  of  the  U.S.  Government  and  no  official  endorsement  should  be  inferred. 
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1 1.  Certification  of  Principal  Investigator 

My  signature  certifies  that  the  above  titled  research  has  been  conducted  in  full  compliance  with  the  HHS  FDA 
Regulations  and  IRB  requirements/policies  governing  human  subject  research.  I  understand  that  a  Progress  Report 
is  required  in  order  to  maintain  continuation  approval  and  any  changes  in  the  study/mclhodology  must  be  approved 
by  the  IRB  prior  to  implementation.  If  the  study  has  never  been  initiated  and  I  am  requesting  termination  (Item  5.c. 
above),  my  signature  certifies  this  request.  If  the  study  is  completed  (Items  5.d.  &  6.c.  above)  and  I  am  requesting 
closure,  my  signature  certifies  that  the  information  provided  on  this  form  represents  an  accurate  final  report. 

Signature  of  Princjpjp  Investigator  Date 

Robert  C.  Zalme,  Colonel,  USAF,  DC 
Oral  and  Maxillofacial  Pathologist 
Keesler  Medical  Center 
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Signature  oi  Civilian  Principal  Investigator 


Mark  S.  Roberts,  MD,  MPP 

Professor  of  Medicine,  Health  Policy  and  Management  and  Industrial  Engineering 
Chief,  Section  of  Decision  Sciences  and  Clinical  Systems  Modeling 
Department  of  Medicine 
University  of  Pittsburgh  School  of  Medicine 
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